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Abstract 

We recently introduced p-automata, automata that read discrete-time Markov 
chains. We used turn-based stochastic parity games to define acceptance of Markov 
chains by a special subclass of p-automata. Definition of acceptance required a 
reduction to a series of turn-based stochastic parity games. Acceptance by general 
p-automata was left undefined as there was no notion of games that supported it. 

Here we generalize two-player games by adding a structural acceptance condi- 
tion called obligations. Obligations are orthogonal to the linear winning conditions 
that define whether a play is winning. Obligations are a declaration that player 
can achieve a certain value from a configuration. If the obligation is met, the value 
of that configuration for player is 1 and is independent of probabilistic choices 
made in the rest of the game. 

We define and show that Blackwell games with Borel objectives and obliga- 
tions are determined by a reduction similar to Martin's proof of determinacy of 
Blackwell games with Borel objectives. We show that for Markov chains with 
Borel objectives and obligations and finite turn-based stochastic parity games with 
obligations there exists an alternative and simpler characterization of the value 
function without going through a Martin-like reduction. Based on this simpler 
characterization we give an algorithm to analyze finite turn-based stochastic par- 
ity games with obligations in exponential time. Finally, we show that obligation 
games provide the necessary framework for reasoning about p-automata and that 
they indeed generalize the previous definition. 

1 Introduction 

Markov chains are a very important modeling formalism in many areas of science. In 
computing, Markov chains form the basis of central techniques such as performance 
modeling, and the design and correctness of randomized algorithms used in security 
and communication protocols. Recognizing this prominent role of Markov chains, the 
formal-methods community has devoted significant attention to these models, e.g., in 
developing model checking for qualitative (1 ,23] and quantitative |4| properties, log- 
ics for reasoning about Markov chains El|6l, and probabilistic simulation and bisim- 
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ulation QISl. Model-checking tools such as PRISM |8| and LiQuor |9| support such 
reasoning about Markov chains and have users in many fields of computer science and 
beyond. 

The automata-theoretic approach to verification has proven to be very powerful 
for reasoning about systems modeled as Kripke structures. For example, it supports 
algorithms for satisfiability of temporal logics (cf. iflOl ). model checking (cf. ifTTI '). 
and abstraction (cf. [12]). 

We have recently introduced p-automata, which are devices that read Markov chains 
as input 1131 . We have shown that p-automata provide an automata-theoretic frame- 
work for reasoning about pCTL model checking, and abstraction of discrete time 
Markov chains. The definition of p-automata is motivated by pCTL fl4l . the de-facto 
standard for model checking Markov chains, and alternating tree automata: they com- 
bine the rich combinatorial structure of alternating automata with pCTL's ability to 
quantify the probabilities of regular sets of paths. The acceptance of Kripke structures 
by an alternating tree automaton is decided by solving turn-based games (cf. |Q3)). In 
that spirit, acceptance of a Markov chain by a p-automaton is decided by solving turn- 
based stochastic games. However, acceptance of p-automata was defined only for a 
subclass, which we called uniform. Intuitively, uniform automata separated between 
measuring probability of regular path sets and setting boundaries on these probabili- 
ties. For uniform p-automata, we showed how acceptance can be decided by a series 
of turn-based stochastic parity games. We could not define acceptance for general p- 
automata as there was no game framework that supported the unbounded interaction 
between measuring probability and setting boundaries. 

Here, we propose such a game notion. In order to support such interaction we aug- 
ment winning conditions in games by adding a structural acceptance condition called 
obligations. A winning condition is a combination of a classical set of winning paths 
and obligations on some of the game configurations. Intuitively, in order to be able to 
use a configuration with obligation, Player has to ensure that the measure of paths 
that satisfy the winning condition from that configuration meets the obligation. If she 
can do that, then the configuration will have the value 1 for her. That is, the value of the 
configuration does not depend on the measure of winning paths obtained in the game 
following the visit to that configuration, only on whether the obligation is met or not. 

Consider the example in Figure [T] Suppose that every path that visits s 2 infinitely 
often is rejecting and every accepting path that visits S4 infinitely often must visit S3 
infinitely often. In addition, in order to use S2, Player makes an obligation to win 
more than | of the paths. In order to use S3, Player makes an obligation to win at 
least 1 of the paths. The obligation of s 2 cannot be met. Indeed, the only way to get 
more than | is to visit the obligation of s 2 again. Thus, in order to fulfill the obligation 
of s 2 , it has to be visited again, recursively, where the recursion is unfounded. As the 
path that visits s 2 infinitely often is rejecting, s 2 cannot be used. On the other hand, 
the obligation of S3 can be met. Indeed, the measure of paths that start in S3 and reach 
s 3 again without passing through s 2 is exactly | . However, whenever reaching s 3 the 
same obligation needs to be met. This can be done by considering the same set of paths 
again. The result has two types of infinite paths. Paths that visit S3 finitely often and 
S4 almost always, whose measure is 0; and paths that visit S3 infinitely often, which 
satisfy the winning condition. 
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Figure 1 : Intuition regarding obligations. Circles denote states of Markov chains. The 
name of the state is written in the top half and the obligation (if exists) is written in 
the bottom half. Probabilities are written next to edges except in case there is a single 
outgoing edge whose probability is 1. 
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Figure 2: A set of measure matters. 

Obligations create a very strong dependence on sets of measure 0. Consider, for 
example, the Markov chain in Figure [2] Suppose that the set of winning paths includes 
all paths. Then clearly, the obligation of configuration S2 can be met. It follows that 
the value of configuration s% is 1. However, by removing from the set of winning 
paths the single path (si • S2) w , whose measure is 0, this changes. The obligation at 
configuration S2 can no longer be met, and the value of si becomes 1/2 (probability 
to reach s 3 without reaching S2). Indeed, in order to use §2, one has to win more than 
half. This implies that in order to use S2 at all, the path that revisits S2 must be used. 
It follows that in order to use S2 once, the path (si • S2) w must be included. As this 
path is now losing it cannot be used. So by removing a set of measure from the set of 
winning paths we change the value of Si. 

Armed with this intuition, we turn to the well definedness of obligation games. Well 
definedness, or determinacy, means that whatever Player loses Player 1 wins and vice 
versa. Formally, it says that the sum of values in a game for Player and Player 1 is 
always 1. This is the most fundamental property that needs to be established for games. 
For almost all types of games used in verification determinacy has never been an issue. 
This relies on Martin's foundational result that Blackwell games with Borel objectives 
are determined 11181 . Blackwell games are two-player games where both players choose 
simltaneously and independently their actions (also known as concurrent games). From 
every configuration a choice of actions determines a distribution over successor con- 
figurations. Blackwell games are general enough so that a simple reduction to them 
suffices to show determinacy for almost all types of two-player games. However, this 
does not apply to obligations. This is apparent as inclusion/removal of a 0-measure set 
can change the value of a game. It follows, that an additional construct is required to 
capture obligations. 

In order to define and show determinacy of Blackwell games with Borel objectives 
and obligations we use a reduction to turn-based games with Borel objectives similar to 
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Martin's proof that Blackwell games with Borel objectives are determined. Intuitively, 
we explicitly add the value to the game and make Player prove that a value can be 
won by showing how it propagates under probabilistic choices made by both players. 
However, there is a crucial component that captures obligations that is not necessary in 
Martin's proof. This ingredient, which we call a discount captures the notion that the 
value for Player may not be achieveable but may be approximated arbitrarily close. 
In order to capture that, Player 1 chooses a small discount to give Player 0, after which, 
Player should not have a problem to show that she can win the required amount. 
The reduction shows that Blackwell games with Borel objectives and obligations are 
determined. 

While the above reduction shows the very general determinacy result, it is not 
amenable to computational analysis as the reduction constructs an uncountable game. 
For computational analysis, we consider the cases of Markov chains with Borel objec- 
tives and obligations and finite turn-based stochastic parity games with obligations. We 
show that in these cases, we can embed the notion of winning into the structure of the 
game by using choice sets. Intuitively, these are the obligations that Player is actually 
going to use. We show that this simpler definition coincides with the definition arising 
from the Martin-like reduction. Based on this simpler and direct characterization, we 
give algorithms that analyze finite turn-based stochastic parity games with obligations. 
We show that deciding whether the value in such a game is at least (or more) than a 
given value r € [0, 1] can be decided in £^ n (intersection of the second level 
of the polynomial time hierarchy) and that the value can be computed in EXPTIME. 
The algorithm identifies a general choice set and calls a solver for finite turn-based 
stochastic parity games for checking the sanity of the choice set. 

We also show that if games with obligations have a finite number of exchanges 
between obligations and no-obligations, then the analysis of the game can be reduced 
to the analysis of a series of Blackwell games (with no obligations). 

Finally, we define acceptance of unrestricted p-automata through turn-based stochas- 
tic parity games with obligations. We show that for uniform p-automata the new defi- 
nition of acceptance coincides with that given in lfl3l . 

Due to lack of space all proofs are included in an appendix. 

2 Background 

For a countable set S let V(S) = {d : S -> [0, 1] | 3T C S such that \T\ G N, Vs i 
T.d(s) — and E^t^s) = 1} be the set of discrete probability distributions (with 
finite support) over S. A distribution d is pure if there is some s G S such that d(s) = 1. 

A countable labeled Markov chain M over set of atomic propositions AP is a tuple 
(S, P, L, s m ), where S is a countable set of locations, P: S T>(S) is a proba- 
bilistic transition, s m G S the initial location, and L : S — > 2 AP a labeling function 
with L(s) the set of propositions true in location s. We sometimes also treat P as 
a function P ; S X S — ¥ [0, 1], where P(s,s') is P(s)(s'). Let succ(s) be the set 
{s' G S | P(s, s') > 0} of successors of s. By definition all Markov chains are finitely 
branching, i.e. succ(s) is finite for all s G S. We write MCap for the set of all (finitely 
branching) Markov chains over AP. A path ir from location s in M is an infinite 
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sequence of locations sqSi . . . with so = s an d P( s i, s i+i) > for all i > 0. 

Given a Markov chain M with set of states S, an open set in S u is a set {w} ■ 
for some w G S*. A set is Borel if it is in the a-algebra defined by these the open sets. 
The measure of every Borel set a is defined as usual in this cr-algebra ifT^l [TTll . We 
denote the measure of a set a as Probjvf (a). 

Blackwell Games A Blackwell game is G = (V, Ao, A\, R, a), where V is a count- 
able set of configurations, Ao and Ai are finite sets of actions, a is a Borel set defining 
the winning set of Player 0, and R : V x A x A\ — > T>(V) is a transition function asso- 
ciating with a configuration v and a pair of actions for both players a distribution over 
next configurations with finite support. A play is an infinite sequence p — vqVi ■ ■ ■ such 
that for every i > there are a° G Aq and a\ G A\ such that R(Vi, a°, a])(vi + i) > 0. 

A strategy for Player is a function a : V + — > 15(^40). A strategy for Player 1 
is defined similarly. A strategy is memoryless if for every w,w' £ V* and v 6 V we 
have cr(wv) = a(wv'). A strategy is pure if for every w € we have cr(u>) is pure. 
Let £ (resp. II) be the set of all strategies for Player (resp. Player 1). 

Each (a, 7r) € S x II from game G and location v determine a Markov chain with 
locations V + . Formally, v(a, ir) = (V + , P, L, v), where L is currently not important 
andfor every w € V* andv,v' e V we set P{vwv') = J2 a <eA SaieAi cr(vwv')(ao)- 
ir(vwv')(ai) ■ R(v' ', ao, ai). 

Sometimes, we may want to start a game from an initial sequence of configu- 
rations, which we call play prefix or just prefix. Let w = Vq ■ ■ ■ v n £ V + be a 
prefix. Then w(a,ir) is the Markov chain ({w} ■ V*,P,L,w), where P(wuv) = 
J2a eA o J2 ai eAi <y{wuv)(a ) ■ ■n(wuv)(ai) ■ R(v, a , a±), foru G V* and v G V. All 
definitions, generalize to this setting. 

The value of (a, ir) for Player from prefix w G {w}-V^ + , is Prob w ( a ^(({w} ■ V u ) D a), 
denoted valo(v(c, 7r), w). The value of w for Player in G is sup inf va\o(w(a, n), w), 

denoted val (G, w). Dually, the value of w for Player 1 in G, denoted vali(G, w), is 
sup inf (1 — va\ (w(a, n), w)). 

Theorem 1 Let G be a game and a a Borel set. Then for every w G V + we have 
valo(G»+vali(G,u;) = l£Z3f. 

The value vali(G, w) can be also obtained by considering the game dual(G) = 
(V, A 1 ,A ,dual(R),V u \a), where dual(i?)(u', ai, a ) = R(v',a ,ax). Formally, 
vali(G, w) — valo(dual(G), w). By definition, for every Markov chain M and every 
measurable set a C V u we have ProbM(a) = 1 — ProbA/(V w \ a). 

Turn-Based Stochastic Games A turn-based stochastic game G is ( ( V, E) , ( V , V\ , V p ) , n, a) 
with the following components. 

• V is a countable set of configurations. 

• E C V 2 is a set of edges such that for every v G V we have | (u, v') G 
is finite. 
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• The triplet (Vq, V\, V p ) partitions V so that Vb is the set of Player configura- 
tions, V\ is the set of Player 1 configurations, and V p is the set of probabilistic 
configurations. 

• k '. Vp — y T>(V) is such that k(v)(v') > if and only if (v, v') £ E. 

• a is a Borel set as before. 

A play is an infinite sequence vqVi ■ ■ ■ such that forall i £ N we have (vi, Uj+i) £ E. 
A strategy for Player is a function a: V* ■ Vq T>(V) such that for all w £ V* and 
v £ Vo we have a(wv)(v') > implies (v, v') £ E. Strategies for Player 1 are defined 
analogously. The type of strategy is determined by the type of game and no confusion 
will arise. As before (<r, 7r) £ E x II determine a Markov chain w(a, tt). Then, the 
value of Player from prefix w is valo(G, w) — sup CTgl; inf^n Prob^^ j7r ) (a) and the 
value of Player 1 from prefix w is vali(G, w) = sup Tgn inf CTe s(l — Prob Ji ,( (T!W )(a)). 

A turn-based stochastic game can be seen as a Blackwell game whose configura- 
tions are of the following types: 

• v is a Player configuration if for every ao £ A and oi, a[ £ A\ we have 
R{v, ao, ai) = R(v, ao, a[) and R(v, ao, ai) is pure. 

• v is a Player 1 configuration if for every ao, a' Q £ Ao and a\ £ A\ we have 
R(v, ao, ai) = a , ai) and ao, ai) is pure. 

• v is a probabilistic configuration if for every ao, a € Ao and a\, a[ £ A\ we 
have R(v, ao, ai) = R(v, a , a^). 

Corollary 1 Let G be a turn-based stochastic game, a a Borel set, and w £ V + . Then, 
val (G, w) = 1 — vali(G, w). 

When V p — the game is simple or just turn based. For turn-based games the set 
V does not have to be countable. In this case it is enough to consider pure strategies, 
which implies that a pair (a, tt) £ S x II induces a unique play w(a, w). Then, the 
value of Player from prefix w is either 1 or 0. Equivalently, Player wins from w if 
there is a strategy a such that for every strategy tt we have w(a, tt) £ a. Otherwise, 
Player 1 wins from w. In this case, we write Wq = {w \ Player wins from w} and 
W\ = {w | Player 1 wins from w}. 

Theorem 2 11911 Let G be a turn-based game and a a Borel set. Then Wq n W\ = 
and W UWi = V+. 

When w £ Wo we also write valo(G, w) — 1 and vali(G, w) = 0. Dually, when 
w £ Wi we write val (G, w) = and val^G, w) = 1. 

We say that a is derived from a parity condition c : V —> [0..k] if for every play 
p = v viV2 ■ ■ ■ £ we havep £ a iff liminf(c(w„)c(u„ + i) . . . ) is even. 

Theorem 3 Consider a finite game G, where a is derived from a parity condition, 
txi £ {>,>}, and r is a rational. 

• If G is a Blackwell game, whether valo(G, w) IX r and vali(G, w) IXI r can be 
decided in PSPACE j |20l/ . 

• If G is a turn-based stochastic game, the values valo(G, w) and vali(G, w) can 
be computed in EXPTIME and whether vali(G, w) txi r can be decided in NP D 
co-NP iEU/. 
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• If G is a turn-based stochastic game, there is a memoryless strategy a that 
achieves the value of G H21\l . That is: 

inf^n Prob w ( CT) ^.)(a) = val (G, w) 

• If G is a turn-based game, whether w € Wq can be decided in NPClco-NP H22V . 

• If G is a countable turn-based game tlien there are pure-memoryless strategies a 
and 7r such that a is winning from every configuration v £ Wq and tt is winning 
from every configuration v € W\ [24]. 

3 Obligation Blackwell Games 

We introduce obligation Blackwell games. These games extend Blackwell games by 
having a winning condition that is composed from a winning set (as in normal Black- 
well games) and a set of obligations. Intuitively, a play is winning for Player if it 
belongs to the winning set. However, whenever meeting an obligation, Player has 
to make sure that the value of the game in that configuration is met. If the obligation 
can be met, the value for Player at the configuration is 1. That is, it is independent 
of actual measure of winning paths enforced by the strategy following the visit to that 
configuration. 

An obligation Blackwell game (OBG) is G = (V, A , A^R, Q), where V, A , A x , 
and R are like in Blackwell games. The goal Q = (a, O), where a C V u is a Borel 
set as for Blackwell games and O : V — > ({>, >} x [0, 1]) U {_L}. The obligation 
function O associates with some configurations the value _L saying that there is no 
obligation associated with this configuration. With other configurations O associates 
an obligation >r or >r stating that Player can use this configuration (and it is going 
to have the value 1 for her) only if she can ensure that the value she can get from this 
configuration meets the obligation. It follows that, recursively, Player has to adjust 
her strategy after every obligation configuration so that the values in all obligations 
are met using plays in ip. For configuration v, if 0(v) ^ _L we call v an obligation 
configuration and if 0(v) = _L we call v a non-obligation configuration. 

We give a formal definition of the value for the players in an OBG through a re- 
duction to a turn-based game similar to Martin's proof that Blackwell games are de- 
termined fl8l . In order to show that the value of Player and Player 1 sum to 1, we 
define the dual game. Dualization of a game consists of changing the roles of the two 
players and switching the goal to the complement. Here, the complementation of the 
goal is slightly more complicated than usual. Consider a game G = (V,Aq,Ax,R,G), 
where = (<p,0). The dual game dual (G) — (V, A\, Aq, dual(i?), dual ((?)), where 
dual( J R)(w,ai,a ) = R(v,a ,ax), dual(£) = (V u \ if, dual(O)), and dual (O) is 
defined below. 

r _l ifo(v) = ± 

dual(0)(» = < >l-r lfO(v)=>r 
[ >l-r UO(v) = >r 

Intuitively, if in G Player has the obligation to achieve more than r with the set (f, 
then the dual player (Player in dual(G)) has the obligation to achieve at least 1 — r 
with the goal set V u \ tp. Syntactically, dual(dual(G)) = G. 
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V= ( 

1 


V+ x (0, 
(w,r,f) 


I]) U (V+x(0,l]x{e}) U 
w G V + ,r G (0,1],/ : succ(to) -> [0,1], and 3d G O(^o) -Vdi G 23(Ai) . ' 
^2 ^2 d o(ao) • rfi(ai) • R(v, o , ai)(^') •/(«')> r 




V = 


(w,r) | either O(w) = _L or 3r'.0(w) = >r'} U {(w,r,e)} 


Vi = 


{(w,r)\Br'.0(w) = > r '} U {(«J,r,/)} 


E= {((w,r),(w,r",e)) \ 0(w) = >r' and < r" < r'} U {((w,r, /), (w •«',/(«'))) I /(«')> °} u 
{((w, r), (w, r', /)) | either 0(w) = _L and r' = r or 0(u>) = >r'} U {((w , f, e), (w, r, /))} 


a = {p£ V w pi}- v G a}, where pj| v is the limit of the projection of p on V + . 



Figure 3: Components of turn(G). 

We now define formally the value of a prefix w E V + by a reduction to a turn-based 
game. We then show that the value is well defined in the sense that the sum of values 
of a prefix w in G and dual(G) is 1. We generalize the function O to apply to prefixes, 
where 0(wv) = 0(v) for every wv G V + and similarly for R, and succ. Consider the 
game turn(G) = ((V, E), (Vq, Vi), a), where the components of turn(G) are given 
in Figure [3] 

There are three types of configurations. Configurations of the form (w, r), where 
O(w) = _L, are illustrated on the left in Figure [4] Such configurations are Player 
configurations, where she claims that the value of prefix w is more than r. From such 
configurations Player chooses a successor configuration (w, r, /), where / is a func- 
tion associating a value to every successor of w that proves that indeed the value at w 
is greater than r. Configurations of the form (w, r), where 0(w) = >r', are illustrated 
in the middle in Figure |4] Such configurations are Player configurations, where, ig- 
noring the value r, she has to prove that the value is greater than r'. Thus, she proceeds 
as above but for the value r' instead of r. Configurations of the form (w, r), where 
O(w) = >r', are illustrated on the left in Figure |4] Such configurations are Player 1 
configurations, where, acknowledging that it may be impossible for Player to achieve 
exactly r' but possible to achieve every r" < r, Player 1 gives Player a discount 
and moves to a configuration (w, r" , e) from which, as above, Player chooses a suc- 
cessor configuration (w, r" , /). Notice, that e is used as a syntactic symbol signifying 
that a discount has been given, it is not a value. Finally, from configurations of the 
form (w,r,f), Player 1 chooses which successor v' of w to follow and proceeds to 
(W, /(</))■ 

By Theorem [2] for every prefix w and for every value r G (0, 1] it is determined 
whether Player wins the game turn(G) from configuration (w, r). 

Lemma 1 For every OBG G and every prefix w, if Player wins from (w, r) in 
turn(G), she wins from every configuration (w,r')for r' < r. Dually, if Player 1 
wins from (w, r) in turn(G), she wins from every configuration (w, r') for r' > r. 

Proof: This can be done by reusing the strategy in turn(G). Essentially, in order to 
show r' < r Player can show r. Dually, in order to show that r' > r is infeasible it is 
enough to show that r is infeasible. □ 

So winning values for Player are downward closed and winning values for Player 1 
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Figure 4: The structure of turn(G). Diamonds are Player configurations and rect- 
angles are Player 1 configurations. A shaded area represents a continuum of edges, 
where every edge is associated with an entry from the continuous domain written next 
to the edge. The fan of discrete edges represents a finite choice, where every edge is 
associated with a possible value from the domain written next to the edge. A dashed 
octagon is either a Player or Player 1 configuration depending on 0(v'). 

are upward closed. It follows that there is a unique value below which Player wins 
and above which Player 1 wins. 

Corollary 2 For every OBG G and every prefix w, there is a value s(G, w) G [0, 1] 
such that for every r' < s(G, w) Player wins from (w, r') in turn(G) and for every 
r" > s(G, w) Player 1 wins from (w, r') in turn(G). 

Notice that Player may or may not win from s(G, w). For a prefix w we define the 
value of w in G as follows. If 0{w) — _L then the value of w in G, denoted val (G, w), 
is s(G, w). If 0(w) = xr then val (G, w) is 1 iff s(G, w) > r or s(G, w) = r and 
ixi = > and it is otherwise. Dually, let vali (G, w) denote the value of w in dual(G). 

Lemma 2 For every OBG and every prefix w, there are distributions d G T>(A ) and 
di G T>(Ai) such that 

s(G,w)= d oi a o) ■ di(ai) ' %> ffl o,ai)(«') • val (G,w • v'). 

Furthermore, for every d' £ T>(Aq) and G T>{A\) the following hold. 
s(G,w) < ^2 ^2 ^2 d o( a o) ■ ^i( a i) • R{w,a,Q,a\){v') • va\ Q (G,w • v'). 

clq£Ao ai^Ai v' Gsuccfu) 

s(G,w)> Y Y Y d o( a o) ■ d ii a i) ■ %,a ,ai)(t)') • val (G,w • v'). 

a £Aa aiGAi t)'Gsucc(u) 
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Proof: Consider the values valo(G,u; • v') for v' G succ(ui). By Von Neumann's 
minimax theorem [23 1 there is an r G [0, 1] and optimal distributions do G V(Ao) and 
di G V(A 1 ) such that 

E E E d (ao)-di{a l )-R(w,a ,ai)(v')v2\ (G,wv') = r 

and for every d' £ V{A$) and every d! x G 22(^1 ) we have 
E E E d o(ao) • d'M ■ R{w 

E E E rf o( a o) • rfi(ai) • -R(w,ao,ai)(w') va| o(G,w • w') < r 

We have to show that r = s(G, w). 

Suppose that r > s(G, w). Let 6 = r — s(G, w) and consider a play starting from 
(w, s(G, w) + 1, /), where / is the function that associates max(0, valo(G, w-v') — |) 
to every successor u' of w. Clearly, the minimax value for / is at least r — |, which 
is larger than s(G,w) + |. By definition of valo(G, w • t/), Player has a winning 
strategy from (w ■ v' ,vz\q(G,w ■ v') — |). It follows that Player also wins from 
(w, s(G, w) + |) contradicting the definition of s(G, w). 

Suppose that r < s(G,w). Let 5 — s(G,w) — r and consider a play starting 
from (w, s(G,w) — |). Consider the configuration (w, s(G,w) — |). In order to 
win, Player has to choose a successor configuration (w, s(G, w) — I, /), where / 
associates at least valo(G, w ■ v') + | with some successor w ■ v' of w. Then, by 
definition of s(G,w), Player 1 wins from configuration (w ■ v',f(v')) contradicting 
the definition of s(G, w). □ 

We are now ready to prove that value is well defined and that obligation Blackwell 
games are determined. 

Theorem 4 Forall prefixes w in an OBG G we have valo(G, w) + vali(G, w) = 1. 

The proof of Theorem |4] is non-trivial. The proof requires Martin's determinacy 
proof style analysis of the uncountable game turn(G), along with new subtleties (for 
example as shown in the example in Figure [2] that measure zero sets could play an 
important role in values of obligation games). 

Proof: For a prefix w, let S(G, w) denote the set of values r such that Player wins 

from (w, r) in turn(G). 

=> We show that if r G S(G, w) then 1 - r $ 5(dual(G), w). 

Suppose that r G S(G, w). That is, Player wins from (w, r) in game turn(G). 
We show that Player 1 wins from (w, 1 — r) in turn(dual(G)) proving that 1 — 
r ^ 5(dual(G), w). Let a be the winning strategy of Player in turn(G). We 
now construct a winning strategy for Player 1 in turn(dual(G)). To distinguish 
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between a prefix of a play in G and prefixes in turn(G) or turn(dual(G)) 
we call the latter two paths. For a path dual(p) in turn(dual(G)) we use the 
strategy a to construct a path p in turn(G) such that whenever dual(p) ends in 
configuration (w' , t) then p ends in configuration (w 1 , r) such that r + t > 1. 
Initially, we start from configuration (w, 1 — r) in turn(dual(G)) and from 
configuration (w, r) in turn(G). That is, both paths are of length one. 
Suppose that the paths p and dual(p) end in configurations (w' , r') and (w' , t'), 
respectively, and that t' + r' > 1. We have the following cases. 

- Suppose that 0(w r ) = _L then (w 1 , r') is a Player configuration in turn(G) 
and (w',t') is a Player configuration in turn(dual(G)). The winning 
strategy a instructs Player to choose some configuration (w',r',f) in 
turn(G). Suppose that Player chooses the configuration (w' , t' , /') in 
turn(dual(G)). By definition, there has to be a configuration v' £ succ(w') 
such that f(v') + f'(v') > 1. We make Player 1 choose (w' ■ v',f(v')) 
in turn(G) and extend the strategy it of Player 1 in turn(dual(G)) by 
choosing (w' ■ v' , f'(v')). 

- Suppose that 0{w') = >r" in G. Then 0(w') = >1 - r" in dual(G). It 
follows that (w' , r') is a Player configuration in turn(G) and (w' , t') is a 
Player 1 configuration in turn(dual(G)). The winning strategy a instructs 
us to choose a configuration (w',r",f) in turn(G). From the minimax 
theorem ||231 it follows that there is a value r'" > r" that is attained for the 
optimal choice do E T)(Ao) such that 

, i-SL J 51 d (a )-di(ai)- R[w\ ao,^)^') •/(«') 

d\£T>{A\)\ ' — ' * — ' ^ — ' / 
\ao£Ao ai£Ai v' £succ(w') J 

is at least r'". LetS = r'" ' -r" '. Notice that l-r" - 5 = 1—r'". Then, from 
configuration (w',t') in turn(dual(G)), Player 1 chooses the successor 
configuration (w' , 1 — r'", e), in effect giving up 5 for Player O's benefit. 
Suppose that Player chooses the successor configuration («/, 1 — r'", /') 
in turn(dual(G)). As above, there has to be a successor v' £ succ(w') 
such that f'{v') + f(v') > 1. Then we make Player 1 choose (w'-v 1 , f(v')) 
in turn(G) and extend Player l's strategy in turn(dual(G)) by the choice 
(wv',f(v')). 

- Suppose that 0{w') = >r" in G. Then 0(«/) = >1 - r" in dual(G). It 
follows that (it/, r') is a Player 1 configuration in turn(G) and (w' , t') is a 
Player configuration in turn(dual(G)). Suppose that Player chooses 
the successor configuration (w 1 , 1 — r", /') in turn(dual(G)). From the 
minimax theorem E3l it follows that there is a value r'" < r" that is 
attained for the optimal choice d\ £ T)(Ai) such that 

inf (EE E *(ao) • di(oi) • R(w', ao, ai)(i/) • /'(«')] 

\p>o£Aq ai£Ai v' Gsucc(u; / ) / 

is at least l-r". Let 5 = r" - r'" . Notice that r" - 5 = r'" . Then, from 
configuration (w' , r') in turn(G), we make Player 1 choose the successor 
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configuration (w' , r'" , e), in effect giving up S for Player O's benefit. Now, 
Player O's winning strategy in turn(G) instructs her to choose a configura- 
tion («/, r'", /). As above, there has to be a successor v' € succ(u>') such 
that /(«') + f'(v') > 1. Then we make Player 1 choose (w' ■ v' , f(v')) in 
turn(G) and extend Player l's strategy in turn(dual(G)) by the choice 
(w'-v',f'(v')). 

Consider the two infinite plays played in turn(G) and turn(dual(G)). Clearly, 
when projecting the two plays on the configurations in V + x (0, 1] that appear 
in them and then on the configurations in V + we get exactly the same play. By 
assumption a is a winning strategy for Player in turn(G). Hence, the limit 
of this projection is in a implying that the strategy constructed for Player 1 in 
turn(dual(G)) is indeed winning. 
<*= We show that 1 - s(G,w) < s(dual(G), w). Notice that if s(G,w) = 1 then 
clearly, 1 — s(G, w) < s(dual(G), w). We consider the case that s(G, w) < 1. 
By definition for every r > s(G,w) we have Player 1 wins in turn(G) from 
(w, r). If t = 1 — r then t < 1 — s(G, w). We show that Player wins from 
(w, t) in turn(dual(G)). 

Consider some value r > s(G,w) such that Player 1 wins from (w,r). We 
show that Player wins from (w, 1 — r) in turn(dual(G)) by proving that 
1 — r e 5'(dual(G), w). We use the difference between 1 — r + s(G, w) and 
1 to give a winning strategy for Player in turn(dual(G)). We use a winning 
strategy w of Player 1 in turn(G) to produce a winning strategy for Player 
in turn(dual(G)). For a path dual(p) in turn(dual(G)) we use the winning 
strategy tt of Player 1 in turn(G) to construct a path p in turn(G) such that 
whenever dual(p) ends in configuration («/,£') then p ends in configuration 
(w', r') such that r' > s(G, w'), Player 1 is winning from p using ir, and t' < 
1-r'. 

Consider a configuration (w, r) such that r > s(G, w). As r > s(G, w) there is 
some r > r > s(G, w) such that Player 1 wins from (w, r). Let tt be the winning 
strategy of Player 1 from (w, r). Initially, we start from configuration (w, 1 — r) 
in turn(dual(G)) and from configuration (w, r) in turn(G). Clearly, (w, r) is 
winning for Player 1, f > s(G, w), and 1 — r < 1 — f. 

Suppose that the two paths p and dual(p) end in a configurations («/,/) and 
(w' , t'), respectively, and that r' > s(G, w), Player 1 wins from p using tt, and 
t' < 1 — r'. We have the following cases. 

- Suppose that 0(w') = _L then (w' , r') is a Player configuration in turn(G) 

and (w' , t') is a Player configuration in turn(dual(G)). 

For every location v' <E succ(w') let u(v') be the following value: 

inf{l, f(v') | (w' ■ v", r', f) € V and tt( p ■ (w' ■ v", r', /)) = v'} 

That is, we consider all possible choices for Player from (w', r'). Such 
a choice includes a function / : succ(w') — > [0, 1]. Then, whenever the 
winning strategy of Player 1 chooses to proceed to v', we record the value 
promised by Player and take the infimum of all these values. 
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By the minimax theorem there are d € T>(A ) and d\ e V(A\) such that 
^2 ^2 ^2 d (ao)-d 1 (a 1 )-R(w',a ,a 1 )(v')-u(v') = r 

and d an d ^i are the optimal distribution choices for both players. We 
show that r < r'. Suppose by contradiction that r > r'. Then, let e = ^^f- 
and consider the function f(v") = max(0, u(v") — e). Clearly, (w 1 , r', /) 
is a configuration in turn(G). However, as n is a winning strategy from p 
the choice ir(p ■ (w' , r' , /)) contradicts the definition of u. So r < r' . 
By assumption if < 1 — r'. Let e = l — r' — t'. Consider now the function 
/' : succ(u/) -> [0, 1] such that f'(v") = 1 - u(v") - f . The minimax 
value of /' in turn(dual(G)) is at least l-f-§>l-r'-§>f'. 
Hence, («/, t', /') is a configuration in turn(dual(G)). 
We extend Player 0's strategy in turn(dual(G)) by choosing configu- 
ration (w',t',f ). Then, Player 1 answers by choosing a successor (w ■ 
v', f'(v')). Notice that it cannot be the case that u(v') = 1. Indeed, in such 
a case f'(v') would be 0. So the path dual(p) is extended by (w 1 , t',f) 
and then (w'-v', f'(v')). 

We now turn our attention to extension of the path p. By the choice of u, 
there is a function / such that (w',r',f) is a configuration in turn(G), 
ir(p ■ (w',r',f)) is (w' ■ v',f(v')), and either f(v') = u(v') or f(v') < 
u(v') + | . So we make Player choose in turn(G) the successor con- 
figuration (w', r',f). Then, Player l's winning strategy n instructs her to 
choose («/ ■ v', /(«'))• 

It follows that f(v') > s(G, w' ■ v'). Otherwise, Player has a winning 
strategy from (w' ■ v' , f(v')) in contradiction with Player l's strategy tt 
being winning. Furthermore, n is winning fromp-(w', r', f)-{w'-v' , /(«'))■ 
Finally, as f'(v') = 1 - u(v') — | and f(v') < u(v') + | we conclude that 
f(v') < 1 - f(v<). 

- Suppose that 0(w') = >r" in G. Then 0{w') = >1 - r" in dual(G). It 
follows that (w' , r') is a Player configuration in turn(G) and (w' , t') is a 
Player 1 configuration in turn(dual(G)). Suppose that Player 1 chooses 
the next configuration (to', £") in turn(dual(G)). 

Now, this is similar to the previous case, as we have to continue from the 
configurations («/, r") in turn(G) and (w 1 , t") in turn(dual(G)) that are 
both Player configurations and t" < 1 — r" . 

- Suppose that O(to') = >r" in G. Then 0(u;') = >1 - r" in dual(G). It 
follows that (w 1 , r") is a Player 1 configuration in turn(G) and (w 1 , 1—r") 
is a Player configuration in turn(dual(G)). 

The winning strategy tt instructs Player 1 to choose configuration (w 1 , r'") 
such that r'" < r". 

As before, this is similar to the first case, as we have to continue from the 
configurations (w',r"') in turn(G) and (it/, 1 — r") in turn(dual(G)) 
that are both Player configurations and 1 — r" < 1 — r'". 
Consider the two infinite plays played in turn(G) and turn(dual(G)). Clearly, 
when projecting the two plays on the configurations in V + x (0, 1] that appear in 
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them and then on the configurations in V + we get exactly the same play. By as- 
sumption 7r is a winning strategy for Player in turn(G). Hence, this projection 
is not in a implying that the strategy constructed for Player in turn(dual(G)) 
is indeed winning. 

□ 

Corollary 3 For every obligation Blackwell game G and every prefix w such that 
O(w) ^ _L we have val (G, w) £ {0, 1}. 

Proof: Consider a configuration v such that 0(w) = cxir. Then, by definition, the 
game turn(G) starting from configuration (w, r') does not depend on the value r'. It 
follows that either Player wins from (w, r') for all r' £ (0, 1] or Player 1 wins from 
(w, r') for all r' £ (0, 1]. It follows that either val (G, w) = 1 or val (G, w) = 0. □ 



4 Markov Chains with Obligations 

We show that for Markov chains the measure of an obligation objective can be defined 
directly on the Markov chain. This direct characterization is generalized later and is 
crucial for the algorithmic analysis for finite games with parity objectives. We intro- 
duce the notion of a choice set, a set of obligations that Player can meet. We then 
show that the definition of a value through a choice set and the definition in Section [3] 
coincide. 

Consider a Markov chain M = (S, P, L, s ln ). Let Q = (a, O) be an obligation, 
wherea C 5" is a Borel set of infinite paths and O : S -> ({>, >}x [0, l])U{_L}isthe 
obligation function. We can think about such a Markov chain as an obligation Black- 
well game where Aq and A\ are singletons. Formally, Gm = (S, {a}, {a}, R, Q), 
where R(s, a, a) = P(s) forall s £ S. As before, we are interested in sequences of 
locations, which correspond to prefixes of plays in Gm- Thus, we refer to them as 
prefixes also here. Let S denote the set of prefixes s ln ■ S* . Let O denote the set of 
locations s £ S such that O(s) ^ _L and O prefixes w ■ s £ S such that s £ O. We 
denote by 0(w) the obligation O(s), where w = w'-s. That is, O is the set of locations 
with a non-empty obligation and the set O is the set of prefixes that end in a location in 
O. Let M — S — O denote the set of locations that have no obligation and N denote 
the set of prefixes S — O. For a prefix w a choice set is C w C On ({w} • S ■ S*). 
That is, it is a set of extensions of w that have obligations. Given a prefix w £ S and 
a choice set C w , an infinite path w ■ y is good if either (a) y — x ■ z, x £ Af* ■ O, and 
w ■ x £ C w , or (b) y £ M u and w ■ y £ a. That is, either the first visit to O after w is 
in C w or O is never visited and the infinite path is in a. Let /3q denote the set of good 
paths of w with choice set C w . Given a choice set C w and a prefix w, the measure of 
Q from w according to C w is: 



\sr M {w,C u 



Prob M (/% 



Prob M (W-^)' 
A choice set C w is good if the following two conditions hold: 
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• Every infinite path it = sq, s\, . . . in M such that n has infinitely many prefixes 
in C w is in a. 

• For every sequence w' G C w we have Msr^ (w', C w ) M r, where 0{w') = xr. 
Let C w denote the set of good choice sets for w. 

Consider a Markov chain M — (S, P, L, s m ) and an obligation Q = (a, O). For a 
prefix w the pre-value of w is 

v(M,0,w)= sup Msr^iu.C). 

cec„ 

Finally, we define the value of w. For a prefix u> such that 0(w) ^ 1 we define 
v(M, £/, iu) to be 1 if v(M, £ , w) M r, where O(w) = oar, and v(M, 5, u>) is other- 
wise. For a prefix w such that 0(w) = _L we define v(M, £/, w) to be v(M, £/, w). 

We note that in a choice set C, if there is some prefix w ^ C such that w E O 
then for every extension w ■ y of w there is no point in including w ■ y in C. Indeed, 
once a certain obligation is not included in C all the obligations that extend it are not 
important. We restrict attention to choice sets that satisfy this restriction. 

We show that for every Markov chain and for every prefix the two definitions of 
value coincide. 

Theorem 5 For every Markov chain M, obligation Q = (a, O), and prefix w € S* we 
have v{M,Q ,w) = valo(Gjy, w). 

The proof of Theorem [5] requires a refined analysis of a winning strategy in the 
uncountable game turn(M) obtained from a Markov chain M. Using this analysis we 
extract from the winning strategy in the uncountable game a witness choice set in M. 

Proof: We show that valo (C?m, w) > v(G, w). 

• Fix e > 0. We have to show that if there is a choice set C that shows the value 
v(G,w) — e then Player can get the value v(G,w) — e in turn(Gj^). This 
proof uses heavily Martin's proof of determinacy of Blackwell games [ 18"]. Fix 
a Markov chain M = (S, P, L, s m ) for the rest of this proof. 

Given a Borel set /3 C S u , Martin defines a tum-based game G m that is slightly 
different to ours. Formally, G m = ((S+ x [0, 1]) U (5+ x F),E, where F 
is the set of functions {/ : S — > [0, 1]} and 

E = {((w,v),(w,f))\ Z s >es p (s,s')-f(s')>v} 

U {{{w,f),(ws,v))\f{s)>v} . 

For a set P C S + , let P t u ' denote P n {w} ■ S + , i.e., exactly all suffixes of 
w in P. Then, Martin's proof shows that > v iff Player wins 

G m from every configuration (w, v') for v' < v. That is, Player announces 
the values she can derive from successors of w and Player 1 chooses a successor 
from which to show the value. Finally, f3 is the set of plays whose projection on 
S + has limit in /?. The strategy of Player forces all infinite plays to be in /3^] 

1 The main difference between the two games (except for no obligations in Martin's version) is as follows. 
In our game the value promised by Player is always slightly below the real value. Accordingly, we require 
that the weighted sum of values of the successors be strictly larger than the promised value. In Martin's 
version the weighted sum of values of successors may be equivalent to the promised values (or larger). 
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We use Martin's result to show that whenever Msr^ // (u>, C) > v then Player 
wins in turn(GM) from (w,v') for every v' < v. Let 6 = v — v'. As 
Msr^f (w, C) > v, then according to Martin's proof Player wins in G m from 
(s, v' + |). We use Player O's strategy in G m to win in turn(GM )• As we play, 
we maintain the requirement in turn(G m ) always below the requirement in G m 
by repeatedly dividing the gap between the values in the two games by 2. It fol- 
lows that in the ith round of playing the two james, the gap between the values 
is Jr. Furthermore, as Player O's strategy in G m is winning it cannot be the case 
that the play created passes through an obligation prefix that is not in C (indeed, 
all continuations from this point are losing in G m ). If on the other hand, a play 
passes through an obligation point that is in C, then the correspondence between 
the game turn(GM) and a new instance of G m from the new obligation point is 
created. Consider an obligation prefix w' such that 0(w') = >v'. By goodness 
of G, Msr^ f (w', G) > v'. In the game turn(GAf) Player 1 moves to a configu- 
ration («/, v" , e), where v" < v' . Thus, we can use the same argument and use 
Martin's game G m to continue the strategy in turn(GM)- Consider an obliga- 
tion prefix w' such that 0(w') = >v'. By goodness of G, Msr^ (w 1 , C) > v'. 
Hence, there is some v" such that v' < v" < Msrj^(w', G) that can be used 
in Martin's game. Finally, consider an infinite play in turn(GM)- If the play 
visits G infinitely often, then by G's goodness, it is winning for Player 0. If the 
play visits G finitely often, according to Martin's result, the corresponding play 
is winning in G m implying that the play is in a. 
In the other direction we show that v(G, w) > valo(GM, w). 

• In the other direction, a winning strategy for Player in the game turn(GM) 
from (w, v) induces a good choice set G. We start by fixing the winning strategy 
a of Player 0. For the sake of this proof we assume that Player always plays 
by this strategy a. Furthermore, from a prefix i»eO such that 0(w) = >v, we 
know that Player 1 can choose every successor (w, v' , e) for v' < v. We restrict 
Player Is choices to those v' such that v' > and v' = v — — for some n E N. 
Thus, when we say configuration is reachable we mean under these choices of 
Player according to a and for choices of Player 1 restricted in >-obligation 
configurations as explained. 

The definition of the choice set G is quite involved as it has to take into account 
the infinitely many different strategies that are involved in showing an obligation 
of the form > r (corresponding to each of the choices v' = v — -). We assume 
that the initial prefix w is an obligation such that 0(w) = >v for the value that 
interests us v. Indeed, this forces Player to be able to win the game (w, v' , e) 
for every v' = v — - for every n E N. The choice set we construct has to factor 
in these infinitely many different strategies. However, the same occurs whenever 
another obligation w' that extends w is reached for which 0(w') = >v" for 
some v". It follows, that the choice set we construct must include the same 
construction for every obligation of the form >v" that is encountered in the 
game. More formally, we have the following. 

Assume that we start from prefix (w, r') such that 0(w) = >v. Let T denote 
the set of prefixes reachable from w excluding w itself. For every w' = w ■ 
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si ■ ■ ■ s n € T, let level(w') denote the number of locations Si such that s$ € 
O and O(si) = >r' for 1 < i < n and r' € (0, 1]. That is, level(w') is 
the number of >-obligations on the way from w to w' excluding w itself but 
including w' (if appropriate). For every i > 1, let Tj C T denote the set Tj = 
{«/ | lev el (w') = i}. It follows that T = Ui>i ^ an d f° r ever y i an d j we nave 
TiDTj — 0. We now restrict attention (by induction) to a subset of the obligation 
configurations that appear in T. 

Consider an obligation w' in T\. For every such obligations there is a mini- 
mal n € N such that it/ is reachable from (w, v — — , e). We call this the rank 
of u>, denoted rank(w). Consider an obligation w' in T^. Let si, . . . , S; be 
the >-obligations on the way from w to w' and let V\, . . . , Vi be the values of 
these obligations. Then, there is a minimal according to the lexicographic or- 
der (m, . . . ,rii) such that w' is reachable from w by Player 1 taking the choice 
(sj, Vj — e) from (sj, v'j) for the appropriate vL As before, we call this the 
rank of w', denoted rank(w'). We say that w' is good if for every >-obligation 
w" on the path from w to w' we have that rank(w") is a prefix of rank(w'). 
That is, whenever w/ is reachable through multiple choices of Player 1, we con- 
sider only the strategy Player 1 used from w' for the minimal choice of discount 
given on all >-obligations on the way to w' . We say that >-obligation prefix 
w" is good if it appears in T and all >-obligation prefixes appearing on the path 
from w to w" are good. That is, if it appears as part of one of the same "minimal" 
strategies. Let C = {w € T l~l O \ w is good}. We note that the definition of C 
does not depend on Gm being derived from a Markov chain. Indeed the same 
definition is used in the proof of Theorem|7] 

In order to show that C is a good choice set we have to prove two things. First, 
that every path that visits infinitely many obligations in C is in a. Second, that 
all obligations in C are met. 

For the first claim we note that an infinite sequence of prefixes in C appears also 
in turn(GjV;f ) an d from a being a winning strategy must be in a. We have to 
show that the obligation of every w' £ C is met. However, for this we can use 
again Martin's reduction. For every prefix w £ C, the strategy of Player in 
turn(Gj^f ) can be used to show a win in G, n from (w, v'), where 0(w) = Mr 
and either IX = > and v' > r or txi = > and v' > r. Essentially, Player O's strat- 
egy in turn(GA/) promises values for each prefix visited in a play. These values 
are larger than the values needed in G m and can be used to construct a strategy 
in G m . If some prefix in O is reached, then clearly the play must be included in 
C as there is a strategy of Player 1 that makes it reachable. Furthermore, every 
infinite play that includes infinitely many prefixes in C can be forced by Player 1 
in turn(GM) showing that it is in a. It follows that in the Markov chain we have 
Usr G M {w,C) x r. 

□ 

The definition above uses the supremum over all good choice sets. We show that 
there is a choice set that attains the supremum. 

Theorem 6 For every Markov chain M, obligation Q = (a, O), and prefix w 6 S 
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Figure 5: An obligation Markov chain with no global good choice set. 
there is a choice set C £ C such that Msr^ (w, C) — s(M, w). 

Proof: Fix a prefix w. There are choice sets {C^jigN such that Msr^ f (u), d) > 
s(G»-^. 

Let C' — Cq. Consider a set Cj+i. Let 

C- +1 = Ci + i — {«/ € Ci + i | 3u>" € Cj for j < £ s.t. u>' = w" ■ y for some y}. 

We set C = \JiLi C[. We show that C is a good choice set. 

Consider an infinite path that visits infinitely many prefixes in C. Clearly, all of the 
prefixes in C belong to the same set C; for some i E N. Hence, by Ci being a good 
choice set, the path is in a. 

Consider a point w' e C. Let £ be the minimal such that w' 6 C[. Then, for every 
extension w" = w' ■ y such that w" ■ y £ Ci we have w" £ C[. Indeed, as w' does not 
have a prefix in Cj forall j < i, so is the case for w" . Then, as Msr^ f («/, Gj) satisfies 
the obligation of u>' then so does Msr^ (w 1 , C). 

Finally, it is simple to see that Msr^ / (w, C) — s(G,w). Indeed, if it were smaller 
than s(G, w) then there is an i such that Msr^ f (w, C\) > Msr^ f (w, C). It must be the 
case that C, includes extensions of w that are not in C, contradicting the definition of 
C. " □ 

It is simple to see that in some cases there is no one good choice set that covers all 
possible prefixes. Consider for example the Markov chain in Figure [5] Suppose that 
the path in which s% appears infinitely often is not in a. Clearly, for every prefix p — 
si ■ ■ ■ sx the pre-value of this configuration is 1, Indeed, the choice set that includes 
exactly p ■ s\ proves that. However, this choice set, establishes the value of p ■ s\ 
as |, which is, as required, more than |, A choice set that shows the values of all 
configurations simultaneously, has to include all prefixes si ■ ■ ■ Si, Thus, the infinite 
path si • si • • ■ is visited infinitely often by this choice set and it cannot be good. 

5 Finite Turn-Based Stochastic Parity Games with Obli- 
gations 

We extend the results from obligation Markov chains to finite turn-based stochastic 
parity games with obligations, and show that the value function in such games has an 
alternate direct characterization using choice sets. The direct characterization is crucial 
to present algorithms to solve finite turn-based stochastic parity games with obligations. 
The simpler definition does not generalize to infinite games, Blackwell games, or more 
general winning conditions. In these more complicated games optimal strategies not 
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always exist. For such games, we need a more elaborate construction that captures 
the winning with e-optimal strategies (and non-existence of optimal strategies). This is 
done by the definition in Section[3] 

We reuse the notation G = ((V, E), (Vo, V\, V p ), k, Q) for turn-based stochastic 
parity games with obligations. Here, Q = (a, O) is a goal and a is derived from 
a parity condition c : V — > [0..k]. Strategies are defined as before. Given a prefix 
w e V + and two strategies a and it, we denote by w(cr, tt) the Markov chain obtained 
from G by using the strategies a and n starting from prefix w. Then, we define the 
value of Player in the prefix w in the game to be 

v(G,u>) = sup inf v(w(a,ir),Q,ui) 

We note that if we extend the definition of a measure of a choice set so that bad 
choice sets give measure for all configurations then the following holds: 

v(G, w) = sup inf supMsr^ f aw) {w,C w ) > 

> sup sup inf Msr^, Jw, C w ) 

<res cv, T e n v ' ' 

This follows from properties of supremum and infimum. In the proof of Theorem [7] 
we actually show that v(G, w) < val (G, w) < sup sup inf (• • • ). Hence, the two are 
actually equivalent. Formally, we show that for every finite turn-based stochastic parity 
game with obligations and for every prefix the two values v(G,w) and valo(G, w) 
coincide. 

Theorem 7 For every finite turn-based stochastic parity game with obligations G and 
prefix w G V + we have v(G, w) = valo(G, w). 

As another illustration of the notion of a choice set consider the game in Figure|6] In 
order to use v\, Player has to win more than h from that configuration. Choosing the 
self loop from ^3 to itself or the edge from U3 to V\ only makes things worse (though, 
each can be chosen a finite number of times). So the only option from v$ is to go back 
to t>2 so that the probability of getting from v-i to V4 is 1. If from V4, Player chooses 
to go to vj the value is | which does not satisfy the obligation of v\. Going from v 4 
to V5, on the other hand, and upon returning from v$ to V4 proceeding to v-j fulfills all 
obligations. Indeed, the value for V\ is 1 as all paths eventually reach 1*5, and the value 
for U5 is I as the loop to itself through ug is winning and the paths from V5 to V4 and 
then on to vj have value j. It follows that a possible choice set for this game is 

G = {vi.v^iv^yviv^v^v^)*}. 

Indeed, Player has a strategy reaching from V\ to V1V2 (v 3^2)*^ 4V5 with probability 1. 
She has a strategy from ViV2(v3V2Yv 4 v^(v 3 6 v^) k to either reach ViV2(v3V2Yv4V 5 (vgV5) k VQ v$ 
or win the parity objectives with probability |. We note that Player uses it's first visit 
to V4 to go to V5, in order to boost the probability needed for the obligation of V\, and 
in subsequent visits goes to vj. 
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Figure 6: Finite turn-based stochastic parity game with obligations requiring mem- 
ory. Diamonds are Player configurations and circles are stochastic configurations. 
Priorities in the range [0..1] next to state define a parity acceptance condition. Only 
configurations vq and v$ have priority 0. 

We show that for a finite turn-based stochastic parity game G, from a winning 
strategy in the uncountable game turn(G), we can construct a regular witness choice 
set C, Using the regular witness and the fact that for finite turn-based stochastic ir- 
regular games finite-memory optimal strategies exist we obtain Theorem|7]. 

Proof: As before, let O = {v G V \ 0(v) ^ _L}, TV = {v G V \ 0(v) = _L}. 
Let Oi = {v G O | c(v) = i} and N t {v G TV | c(v) = i} be the obligation and 
non-obligation configurations with priority i. Similarly, let 0>i = {v G O | c(v) > i} 
andTV>i = {v G O \ c(v) > i}. 

We show that v(G,w) > val (G, w). 

• Consider a winning strategy for Player in turn(G) that starts in (w,r). We can 
extract from it a set G of obligations that are used. This is done just like in the 
proof of Theorem [5] Clearly, every path that visits infinitely many prefixes in G 
is in a as it appears also in turn(G) as before. We modify G to a set C that we 
can show is good. We construct C by induction on the number of obligations 
passed on the way from (w,r). 

Consider a prefix w'eCU { w }- Let ob(w') be the set of obligations w" G G 
such that w" = w'-w'" and w'" £ TV* -O. That is, ob(w') is the set of obligations 
directly reachable from w' without passing through other obligations. Further- 
more, annotate every prefix w" in ob(w') by the minimal priority occurring in 
it/", where w" — w' ■ w" 1 and vJ" £ TV* ■ O. For every prefix ai'eCU {w} we 
define a set obs(w') C V x [0..fc], where [0..k] are the priorities of the parity con- 
dition. Formally, obs(w') is the set of pairs (V, i') such that some w" G ob(w') 
is annotated by i' and the last configuration in w" is w'. 

Let TV and 0^ denote the configurations in G whose priority is i. Let TV>i and 
0>i denote the configurations in G whose priority is at least i. 
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We now construct C by induction. We label every prefix p' € C by a prefix 
peC that is the reason for inclusion of p' in C. Consider the configuration w. 
By construction, for every (v, i) € obs(w) there is a prefix € C such that 

w (u,i) € ob(w) and w^.i) ends in v. We add to C" all the prefixes w ■ p, where p 
is in the following set (restricted to those reachable from w): 

(J (W>j • Mi ■ Af>i ■ M u A/> 4 • (Oi n {«})) 

(u,i)Gofrs(i(;) 

Furthermore, every prefix w-p for p g (A/^ • M •7V> i • {w}UA/'> i • (Oj n{u})) is 
labeled by w ■ We now continue by induction. Consider a prefix pf e C 

that is labeled by prefix p e C. By construction, for every (u, i) g obs(p) there 
is a prefix P(„,i) € C such that P(„,i) g ob(p) and P(„,i) ends in v. We add to 
C all the prefixes p' ■ p", where p" is in the following set (restricted to those 
reachable from p'): 

(J (N>i ■ M ■ N>i ■ {v} u A/> 4 • (Oi n H)) 

(u,i)eo6s(p) 

Furthermore, every prefix p' -p" for p" g (Af* l -Af { ■Af* l ■ {v}UAf^ i ■ (O l f){v})) 
is labeled by p ■ ■ 

This completes the construction of C. We have to show that C is a good choice 
set. That is, every infinite path in G that visits infinitely many configurations in 
C is fair and the strategy of Player in G establishes all the obligations posed 
byC". 

The fact that every infinite path in G that visits infinitely many prefixes in C 
is fair can be deduced by following the labels in C of prefixes in C. Consider 
such an infinite sequence of prefixes po,Pi, ■ ■ ■ in C and their respective labels 
wo, Wi, . . . from C. By construction, the minimal priority visited in the exten- 
sion of pi to Pi+\ is the minimal priority visited in the extension of Wi to Wi+i. 
Furthermore, the sequence w ,Wi, . . . corresponds to a path in G that visits in- 
finitely many configurations in C. As C is obtained from turn(G), it follows 
that the limit of wq, w\, . . . is fair. That is, the limit of wo, w\, . . . satisfies the 
parity objective. We conclude that the limit of po, pi , . . ■ is fair as well. 
We now have to show that all obligations in C are met. Consider a prefix p' g C 
labeled by prefix p g C. Both p' and p end in the same configuration v g V 
of G. It follows that the obligation 0(v) is fulfilled in turn(G). Assume that 
0(v) = >r. It follows that Player wins in turn(G) from (p, r') for every 
r' < r. Recall the sets ob(p) C C and obs(p) C V x [0..k]. The winning in 
turn(G) from (p, r') for every r' < r can be translated to a win in G for the goal 
ob(p) ■ V" for every r" < r, where G is the game obtained from G by Martin's 
reduction. Thus, the value of ob(p) ■ U(afl Af u ) in G is r. We now consider 
the following goal 7 in G: 

1= (J - M h ■ M) u {MU ■ (Oi n W))) u (anF) 

(v,i)£obs(p) 
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In particular, 7 contains at least all the extensions p' such that p ■ p' e ob(p) 
as well as a n A"". Furthermore, 7 can be translated to a parity goal in G by 
including a simple monitor for the minimal parity encountered along the path. 
As reach(ob(p)) U(afl TV") C 7 it follows that the value of 7 in G is at least r. 
However, values in finite turn-based stochastic parity games are attained. That is, 
there is a strategy for Player such that the value of 7 according to this strategy is 
at least r. It follows that by using this strategy Player can ensure the obligation 
of p in G. The case that 0(v) = >r is simpler, as Player wins directly from 
(v, r) in turn(G). 

It follows that C is a good choice set and that Player has a strategy that ensures 
that all obligations in C are met. 
In the other direction we show that v(G, to) < valo(G, to). 

• Suppose by way of contradiction that r = v(G, to) > valo(G, to). Let t < r 
be such that t > val (G, w). By definition of val (G,w), Player 1 wins in 
turn(dual(G)) from (to, 1 — t). This winning strategy induces a good choice 
set T just like in the previous proofs. Note that this set is good for Player 1. 
Thus, every path that visits infinitely many configurations in T does not satisfy 
the acceptance condition. As in the other direction of the proof, the set T can be 
extended to a good choice set T' such that Player 1 has a strategy to enforce all 
obligations in T" (in the dual game). We show that T' proves that the value of w 
in G cannot be r. Fix a strategy a of Player in G. We show that the strategies 
of Player 1 in G that enforce the obligations in T" (in the dual game) induce a 
strategy 7r £ II such that every choice set in G a> „ that shows the value r cannot 
be good. Notice, that the set O is the same in G and dual(G). Hence, there is a 
strategy it for Player 1 in G that achieves the value 1 — t for Player 1 . Consider 
the strategy a and assume that it achieves the value r for Player in G for some 
choice set G. Then, as 1 — t + r > 1, and A/"" n a and A/"" n a are disjoint, 
it follows that G and T' have a non-empty intersection such that the strategy a 
reaches G n T'. 

We now proceed by induction. Consider a configuration to' € G n T". There are 
two cases, either 0(w') = >r' or O(w') = >r'. 

- Suppose that 0(w') = >r'. In this case, the obligation of w' in dual(G) 
is >1 — r'. It follows that there is some value r" < r' such that the value of 
{T ■V U )\J {N u n 5) in G for Player 1 is 1 - r". Then, as 1 - r" +r' > 1, 
and A"" n a and A" w n a are disjoint, it follows that G and T have a 
non-empty intersection such that the strategy a reaches C fl T'. 

- Suppose that O(w') — >r'. In this case, there is a value r" > r such that a 
must attain the goal (G • V u ) U (A"" n a) with probability r". At the same 
time Player 1 can force the goal (T' ■ V u ) U (A"" n 5) with probability 
1 — r'. As r" + 1 — r' > 1 the sets G and T' have a non-empty intersection 
such that the strategy a reaches C fl T'. 

Continuing by induction we create a path that visits infinitely many configura- 
tions in T' and in G. It follows that G cannot be a good choice set. 

□ 

Corollary 4 For every finite turn-based stochastic parity game with obligations G 
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and prefix w G V + , there are strategies a € S and tt € II swc/z f/zaf v(G, w) = 
v(u>(<7, 7r), £/, u>). Furthermore, for every strategy a' € S ant/ 7r' € II we /lave 

\z(w(a', it), w) < v(G,w) < v(w(a,Tr'),g,w) 

Proof: This follows from the proofs of Theorems [5] and [7] Consider a configuration 
w. Suppose that v(G, u>) = r. Then, for every n there is a strategy er„ such that for 
every tt € II the value v(G CT?l; ,r(u>), £?, w) > r — A. Furthermore, there is a good 
choice set C n such that the goal (C n ■ V u ) U (A/"" n a) is enforced with probability 
at least r — ~. As in the proof of Theoremplthe different choice sets {C n } n> o can 
be combined to a single choice set G. Furthermore, the choice set G has a simple 
structure as in the proof of Theorem [7] It follows that Player can enforce the goal 
(C • V^) U (N" n a) with probability larger than r — i for every n. As G is finite it 
must be that (G • V") U (7V W n a) can be enforced with probability r. 

The proof that Player 1 also has an optimal strategy is similar. □ 

6 Algorithmic Analysis of Obligation Games 

We give algorithms for solving obligation Blackwell games in two cases. First, in 
case that in every path in the game, the number of transitions between an obligation 
configuration and a non-obligation configuration is bounded. In this case, we show 
that obligation Blackwell games can be reduced to a sequence of turn-based stochastic 
games. Second, in case that the game is finite and the winning condition is a parity 
condition. In this case, we give an exponential time algorithm for computing the value 
of the game. 

6.1 Reduction to Stochastic Games 

Essentially, this is the solution adopted in lfl3l for solving acceptance of uniform p- 
automata. We partition the game to regions where there are no transitions between 
obligation configurations and non-obligation configurations. A region that consists 
only of non-obligation configurations can be thought of as a stochastic game. A region 
that consists only of obligation configurations can be thought of as a turn-based (non- 
stochastic) game. More formally, we have the following. 

Consider an obligation Blackwell game G = (V, Aq, A\, R, Q), where Q — (a, O). 
We say that a configuration v is pure if for every a,Q E Aq and a\ G A\ we have 
R(v, Oo, ai) is pure. We say that the game is uniform if the following holds. There 
is a partition of V such that for every i we have, either (i) for every v € V{ 

we have 0(v) = _L or (ii) for every v G Vi we have 0(v) ^ _L or v is pure. We say 
that Vi < Vi' if there are some v G Vi, v' G Vy, ao G Ao, and a\ G A\ such that 
R(v, ao, ai)(v') > 0. The partition must also satisfy that every chain according to < 
is finite. 

Theorem 8 The computation of the value of a uniform obligation Blackwell game G 
can be reduced to the solution of multiple Blackwell games. 
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Proof: Let {Vi}jgN be the partition of the game G. By assumption, consider a set Vi 
such that there is no other set Vy such that Vi < Vy . Consider a prefix w — w' ■ v such 
that v € V,. Clearly, the extension of this prefix to a play in G remains forever in V,. 

Suppose that forall v G Vi we have 0(v) = _L. Let G' = (V*,A Q , A ll R, a) be 
the game obtained from G by restricting attention to configurations reachable from w. 
The game G' is a normal Blackwell game and hence the value of every configuration 
in {w} ■ V* is well defined. 

Suppose that forall v € Vi we have 0(v ) ^ _L or v is pure. Consider the turn- 
based game G' = ((V* U V* x 2 V ,E), (V* ,V* x 2 y ), a'), where we restrict V* to 
configurations reachable from w and E and a' are as follows. Consider a prefix u' • u' 
and a set 5 C V . If 0(V) = -L, we say that S is possible from u' • 1/ if there is 
d Q e T>(A ) such that forall di G T>(A{) we have 

E E E ^(«',a ,ai)(«")^P, (1) 

oo6A aiGAi u"£S 

where O(w') = cx] p. If t/ is pure, we say that S is possible from w' • w' if there is cto G 
ylo such that forall a\ G Ai the unique configuration w" such that R(v, a,Q, ai)(v") = 1 
is in S. Notice, that this is like considering a pure configuration as having the obligation 

>L 

• E= {(w'-v',(w'-v',S)) I 5 possible from w' ■ v'}U{((w',S),(w' -v')) \ v' G 
V}. 

• a' includes all infinite paths such that the limit of their projection on V * is in a. 
This is in effect equivalent to the reduction to turn(G) when restricted to {w} ■ V*. 

Consider now a set Vi and a configuration w = w' ■ v such that » 6 7;, Suppose, 
by induction, that for all configurations u ■ v' such that v' G Vy for V L < Vy a value 
has already been computed. 

If for every v € Vi we have 0(v) — _L, then a similar reduction to a normal 
Blackwell game by plugging in the value of precomputed configurations gives the value 
of all configurations in {w} ■ V*. 

If for every v G Vi we have 0(v) ^ _L, then a similar reduction to a turn-based 
game can be done. This time, value of precomputed configurations has to be combined 
in the small minimax games as in Equation[T] □ 

We note that this is a "meta"-algorithm. Consider a uniform obligation Blackwell 
game G and the partition V\ , . . . , V n showing that it is uniform. Suppose that every 
V, reduces to a Blackwell game that can be analyzed algorithmically. Then, from 
Theorem[8] the game G can be analyzed algorithmically. 

6.2 Finite Turn-based Stochastic Parity Games with Obligations 

We show that values in finite turn-based stochastic parity games with obligations (POG, 
for short) can be computed in EXPTIME and decision problems regarding values can 
be solved in n n^. 

We give a nondeterministic algorithm for finding a maximal choice set, which calls 
the computation of values in stochastic parity games as a subroutine. Then, the value 
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Figure 7: Illustration of a dependency. Priorities in the range [0..4] next to state define 
a parity acceptance condition. 

of a configuration in the game can be computed by computing the value of reaching the 
choice set computed by the algorithm or winning the parity condition without reaching 
other obligations. By results of previous sections, dualization of the game gives the 
value of the opponent. It follows that the value can be computed also in co-NP with an 
oracle in co-NP. 

We now give an algorithm that decides and computes values in G. A dependency 
for v E O is either C v = _L or C v C (Ox [0..fc]). That is, C v is either undefined 
or a (possibly empty) set of pairs of obligation configurations annotated by priorities. 
A game dependency is a set {C v } ve o- A game dependency is good if the following 
conditions hold: 

1. If for some v € O we have (V, i) £ C v then C v i 7^ _L. 

2. For every infinite sequence (vq, io), (vi, ii), . . . such that for every j we have 
(vj+i, ij+i) E C v . the minimal priority occurring infinitely often in io, i\, . . . is 
even. 

3. For every v E O such that C v ^Iwe have val (G", v) cxi r, where 0(v) = t<r 
and G' is the game G considered as a turn-based stochastic game with the goal 

/ {NZi-Mi-NU-W'V") u \ 
|J (N>i ■ (Oi n {v}) ■ V u ) u 

(v,i)ec v \ (an TV") / 

We illustrate the notion of a dependency using Figure[7] State si has an obligation 
of I . In order to fulfil this obligation it has to reach itself again. However, the paths 
(sisJs4S6)" have a minimal priority of 1 occurring on them and are losing. It follows 
that the paths that can be used are (sisj S4S5) 1 *' , (515^5455)", and (sisj S4S6)" . Thus, 
si depends on reaching si with minimal priority (through S3) and on reaching s\ 
with minimal priority 2 (through S2 and S5). This satisfies the three conditions as (1) 
si has a defined dependency, (2) every cycle visits either the minimal priority or 
2, and (3) the probability of reaching si with minimal priority is i, the probability 
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of reaching si with minimal priority 2 is \, and the probability of not reaching s\ 
(on paths that satisfy the parity condition) is 0. So the total probability is §, which 
fulfils the obligation of s\. Adding an obligation of >i at S4, changes the dependency. 
Now, Si depends on reaching S4 with priority or 2 and S4 depends on reaching 
with priority 3. However, if the obligation of s 4 is set to >~, then there is no good 
dependency. Indeed, this would mean that whenever S4 is reached the path through sq 
must be taken. In particular, the path from si through S2 can no longer be used leading 
to the obligation of si not being fulfilled. The dependency for the game in Figure [6] 
is vi depends on reaching v$ with priority 1 and W5 depends on reaching itself with 
priority 0. 

The nondeterministic algorithm is as follows. We guess a game dependency {C v } v <£0 ■ 
The size of {C v } v ^o is polynomial in |V|. We check that {C v } v ^o is good by doing 
the following. First, checking that if (V , i) € C v then C v > 7^ _L can be completed in 
polynomial time by scanning all the sets C v . Second, checking that all cycles induced 
by C v have a minimal even parity in them can be completed in polynomial time by 
drawing the graph of connections between the different configurations in O for which 
C v 7^ _L and searching for a cycle with minimal odd priority. Third, ensuring that 
the values in the different turn-based stochastic games fulfill the obligations can be 
completed in NPflco-NP by Theorem|3] Finally, consider the goal 7': 

7' = (W* • {v : V c _L} • V") U (aflF) 

We evaluate whether val (G, w) M r by checking whether val (G", w) ix r, where G' 
is the turn-based stochastic game obtained from G by considering the goal 7'. This 
can be checked in NPnco-NP. To compute the value valo(G, w) we compute the value 
of w in G' . This can be computed in EXPTIME. Notice that the values of 7' in G' 
correspond to the value s(G, w) and not valo(G, w). For obligation configurations we 
must compare the result with the required obligations. If the obligation is met, the 
value valo(G, w) is 1. Otherwise, it is 0. Overall, the decision problem is in (~l n^, 
and the values can be computed in EXPTIME. 

We apply the algorithm on the example in Figure [7] As analyzed above, the depen- 
dency for si is (si , 0) and (si , 2). This proves that the value for all configurations is 1. 
Indeed, for every configuration in the game the probability of reaching si at least once 
is 1. Once si is reached for the first time, the more complex reliance on the choice set 
that is extracted from the dependency is required. Applying the algorithm on the game 
in Figure[6]we see that the value of V\, V2, V3, V4, and vq is 1 as from them Player can 
reach 175 with probability 1. The pre-value of U5 is | as it reaches itself with probability 
I and wins the parity condition (reaching w§) with probability j. As this matches its 
obligation its value is 1. 

Correctness of the algorithm follows from the following three Lemmas. 

Lemma 3 There is a memoryless winning strategy in turn(G). 

Proof: According to the proof of Theorem|7]from every obligation used as part of the 
winning strategy in turn(G), there is a simple goal that leads to the next frontier of 
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used obligations. Namely, given the sets ob(p) and obs(p) the goal is: 

7= (J P>rM-^rW)u(^ r (0,nW)))u(anr) 

The structure of this goal implies that there is a strategy with memory linear in the 
number of priorities in the game that achieves an optimal value for this goal. However, 
an obligation configuration v E O may appear infinitely often in turn(G), each time 
using a different strategy. 

Consider now all the possible strategies for Player in G with a goal 7 as above 
with memory bounded by the number of priorities. Clearly, the number of such strate- 
gies is finite. In particular, for every obligation configuration v £ O there is a finite 
number of strategies that are used in turn(G). We now construct a finite parity game 
G' based on these strategies. For every prefix p ■ v such that v £ O used in turn(G) 
add a Player configuration v to G'. For every strategy a that is used from the prefix 
p ■ v in turn(G) we add a Player 1 configuration a to G' . For every pair (V, i) such 
that v 1 E O and i is a priority such that application of a from p ■ v reaches a config- 
uration v' with priority i being the minimal visited along the way we add the Player 
configuration (a, v' , i) to G' . We add edges to G' as follows. From configuration v we 
add edges to all strategies a used from p ■ v for some p. From strategy a we add edges 
to all triplets (a, v', i). From configuration (a, v ' , i) we add an edge to v'. We set the 
priority of (a, v' , i) to be i and priorities of all other configuration to be the maximal 
possible priority. 

The game G' is a finite parity game and we know that Player wins G' based on 
the combination of the winning strategies in turn(G). It follows from Theorem [3] that 
there is a memoryless winning strategy for Player in G'. However, a memory less 
winning strategy in G' induces a unique choice of a strategy from every obligation 
configuration in turn(G) leading to a memoryless winning strategy in turn(G). □ 

Lemma 4 An obligation configuration v fulfills valo(G, v) = 1 iff there is a good game 
dependency {C v i} v i^o such that C v 7^ 0. 

Proof: The existence of a good game dependency clearly shows that the obligation of 
v can be met. 

In the other direction, if the obligation of v can be met, this means that Player 
wins in turn(G) from (v, r') for every r' £ (0, 1]. Furthermore, a choice set of a very 
particular form can be extracted as in the proof of Theorem|7] According to Lemma|3] 
Player has a memoryless winning strategy in turn(G). We note further, that in 
turn(G), if an obligation configuration (w, r') occurs, the game below (w,r r ) does not 
depend on the value r'. Thus, if two obligations w ■ v and w' ■ v and the configurations 
(w, r) and («/, r') occur in turn(G) the extension of the game below both is identical. 
It follows, that the memoryless strategy behaves exactly the same from all obligations 
w ■ v and w' ■ v for the same obligation v. Then obs(w ■ v) — obs(w' ■ v) for all 
w, to' e F. So for an obligation v appearing in turn(G) we can use the set obs(w ■ v) 
for some prefix w as the dependency C v . For every obligation v' not appearing in 
turn(G) we set C v i = _L We have to show that this induces a good game dependency. 
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First, if we have (V, i) 6 C v then it follows that some prefix w' ■ v' is reachable from a 
prefix w ■ v. Thus, C' v must be defined. Second, every cycle in {C v } v ^o with minimal 
odd priority corresponds to an infinite path in the good choice set with a minimal odd 
priority, which is impossible. Third, it must be the ca.se that valo(G", ^) tx] r, where G f 
is obtained from G by considering the goal 7: 

7= IJ (CA/">i • M • •{«'}• V u ) u {M>i ■ (Oi n {«}) • V u )) u(anM u ) 

(v>,i)ec v 

Indeed, this is the exact construction of the choice set from turn(G) as in the proof of 
Theorem]?] where it is proven that it is also good. □ 

Lemma 5 For every configuration v, valo(G, v) — r iff there is a good game depen- 
dency {C v '} v ' € o such thatva\(G' , v) = r, where G is obtained from G by considering 
the goal 7. 

7 = (.Af* • {v : V c 96 ±} ■ V") U (a n TV") 



Proof: As before, if val(G", v) — r then clearly valo(G, v) > r. In the other direction, 
we consider all the obligations appearing in the choice set showing that val (G, v ) = r. 
According to the previous lemma, these obligations require a good game dependency. 
Finally, the value valo(G, v) is exactly the reachability of the good choice set or win- 
ning parity without reaching obligations. □ 

Theorem 9 For a POG G and a prefix w € V + , the values valo(G,ui) and vali(G,w) 
can be computed in EXPTIME and whether valo(G, w) CO r can be decided in nllf . 



7 p-Automata 

We recall the definition of p-automata in fl3l . Here we remove the restriction of uni- 
formity. To simplify presentation we remove the notion of * -transitions. In a longer 
version of this paper we will show how to generalize obligation Blackwell games to 
include separation restrictions as well. Using this separation, all concepts developed 
so far generalize to ^-transitions. In essence, disjoint value functions, like the ones 
used in fl3l . can be composed in obligation Blackwell games without causing major 
difficulties. 

We assume familiarity with basic notions of trees and (alternating) tree automata. 
For set T, let B + (T) be the set of positive Boolean formulas generated from elements 
t G T, constants tt and ff, and disjunctions and conjunctions: 

(p,ip ::=t I tt I ff I ipV V I (2) 

Formulas in B + (T) are finite even if T is not. 

For set Q, the set of states of a p-automaton, we define term sets [Q]> as follows. 

[Q]> - {[?]** I ? e Q,m e {>, >} lP e [0, 1]} 
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Intuitively, a state q 6 Q of a p-automaton and its transition structure model a 
probabilistic path set. So [g]x p holds in location s if the measure of paths that begin 
in s and satisfy q is tx] p. 

An element of Q U [Q] > is therefore either a state of the p-automaton, or a term 
of the form [gj^p. Given ip e B + (Q U [Q]>), its closure c\(ip) is the set of all 
subformulas of ip according to (|2ji. For a set $ of formulas, let cl($) = U^e* c '(v)' 

Definition 1 A p-automaton A is a tuple (E,Q,5,(p ln ,a), where £ is a finite input 
alphabet, Q a set of states (not necessarily finite), 5 : Q x £ — > B + (Q U [Q]>) ?/ie 
transition function, ip m 6 -B+dQ]^) f/ie initial condition, and a a parity acceptance 
condition. 

In general, p-automata have states, Markov chains have locations, and games have 

For every AP, p-automata A — (2 ,Q,5,ip ln ,a) have MCap as set of inputs. 
For M — (S, P, L, s m ) 6 MCap, we define whether A accepts M by a reduction to 
a turn-based stochastic parity game with obligations. The language of A is C{A) = 
{M € MCap I A accepts M}. 

We construct a game Gm,a = ((V, E), (Vq, Vi, V p ), k, Q). A configuration of 
Gm,a corresponds to a subformula appearing in the transition of A and a location in 
M. Configurations with a term of the form \q\^ p correspond to obligations. All other 
configurations have no obligations. The Markov chain is accepted if the configuration 
<y n , s in ) has value 1 in G M<A . 

Formally, we define Gm.a as follows. Let Gm,a = ((V, E) 7 (V , Vi, V p ), k, Q), 
where the components of Gm^a are as follows. 

• V = Sx cl(5(Q,E)). ' 

• V Q = {(s, Vi) 2 )\seS and Vi V V2 G cl(5(Q, £))}. 

• Vi = {{a, ipiAip 2 )\s € S and V> 2 A V2 G cl((S(Q, £))}. 

• V p = firx (QU[Q]>). 

• The set of edges E is defined as follows. 

£={((*> ¥>i A<p 2 ),(s,<pi)) |i€ {1,2}} U 
{((a,¥>i V<p2),(s,<Pi)) |<6 {1,2}} U 
{((s,q),(s',5(q,L(s))))\s' €succ(s)} U 
{((s, Mxp), (s',S(q, L(s)))) I s' G succ( S )} 

. k((s, q), (s', L(s)))) = M Mp ), («', «%, = P(s, s')- 

• £/ = (5, O), where 

- For q E Q and p£ [0, 1] we have a(s, g) = a(q), a(s, = For 
every other configuration c we set a(c) to the maximal possible priority. 

- For q E Q and p G [0, 1] we have 0(s, [gjixp) = ixp. For every other 
configuration c, we have O(c) = _L. 

Theorem 10 Given a p-automaton A — (2 Ar , . . .), its language C(A) is well defined. 
Proof: This follows from Theorem|4] □ 
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Theorem 11 Given a finite p-automaton A and a finite Markov chain M, we can de- 
cide whether M € C(A) in time exponential in the number of states of A and locations 
ofM. 

Proof: This follows from Theorem[9] □ 

The definition in lfl3l restricts attention to uniform p-automata. Such automata 
restrict the cycles in the transition graph of p-automata. We recall the definition of 
uniform p-automata. In doing so, we differentiate states q' appearing within a term 
in [Q]> (bounded transition) from q' appearing "free" in the transition of a state q 
(unbounded transition). In this way, a p-automaton A = (£, Q,S, . . .) determines a 
labeled, directed graph Ga = (Q' , E, E b , E u ): 

Q' = QUcl(*(Q,E)) 

E = {(ifi A ip 2 , (Pi), (ipi V tp 2 , Pi) I Vi 6Q'\Q, 
i e{l,2}} U{(q,8(q,a))\qeQ,treY} 
E u = {(ip A q, q), (q A tp, q),(<pV q, q), (q V tp, q) \ 

<peQ', q eQ} 

E b = {(Mixp,g) I Mxp g [Ql>} 

Elements (<^, g) e B u are unbounded transitions; elements (tp, q) G Eb are bounded 
transitions; and elements of E are called simple transitions. Note that E, E u , and Eb 
are pairwise disjoint. Let <p <a (p iff there is a finite path from ip to (p in _E U i?6 U £7 U . 
Let = be <a H and ((95)) the equivalence class of with respect to =. Each ((ip)) 
is an SCC in the directed graph Ga- 

Definition 2 A p-automaton A is called uniform if: 

• For each cycle in Ga, its set of transitions is either in E U Eb or in E U E u . 

• There are only finitely many equivalence classes ((ip)) with ip G Q U c\(5(Q, £)). 

Then, A is uniform, if the full subgraph of every equivalence class in contains 
only one type of non-simple transitions. Also, all states q' E Q or formulas tp occurring 
in S(q, a) for some q € Q and a € E can be classified as unbounded, bounded, or 
simple - according to SCC ((g)). 

Intuitively, the cycles in the structure of a uniform p-automaton A take either no 
bounded edges or no unbounded edges. Uniformity allowed to define acceptance of 
input for p-automata through the solution of a sequence of stochastic games. 

Theorem 12 Definition^extends the definition of uniform p-automata in H13V . 

Proof: The obligation Blackwell game resulting from the composition of a uniform 
p-automaton with a Markov chain is a uniform tum-based obligation game. From 
Theorem [8] it follows that its value is obtained from solving a sequence of turn-based 
stochastic games and turn-based games. This gives rise to exactly the definition of 
acceptance through a sequence of turn-based stochastic games and turn-based games 
as in 02). □ 

Based on dualization of Blackwell games with obligations, one can show that dual- 
ization of p-automata (see [13]) leads to complementation. Closure under intersection 
and union is simple. 
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8 Conclusions and Future Work 



We introduced obligations, a structural winning condition in games that complements 
winning conditions on paths. We show that Blackwell games with Borel objectives and 
obligations are well defined. We then present a simpler definition of value for Markov 
chains with Borel objectives and obligations and for finite turn-based stochastic parity 
games with obligations. Based on the simpler definition we give algorithms for ana- 
lyzing finite turn-based stochastic parity games with obligations. We then use games 
with obligations to define acceptance by unrestricted p-automata, showing that the new 
definition generalizes a previous definition for uniform p-automata. 

This is one of the rare cases in games that arise in verification that determinacy of 
games does not immediately follow from Martin's result that Blackwell games with 
Borel objectives are determined. The proof of determinacy uses elements from Mar- 
tin's determinacy proof but introduces new concepts that were not needed in that proof. 
These new concepts are required due to the more elaborate nature of games with obli- 
gations. 

Our work gives rise to many interesting questions. For example, optimality of 
algorithms for analysis of games with obligations is not clear. We establish that solution 
of finite turn-based stochastic parity games with obligations is in n n^. It is an 
interesting question whether they can be solved in NPHco-NP as for games without 
obligations. In particular, our approach, if generalized to Streett and Muller games, 
would result in algorithms in NEXPTIME. The huge difference in complexity between 
Streett and Rabin objectives suggests that other techniques for solving such games may 
be possible. We have not considered at all algorithms for infinite games, concurrent 
games, or partial information games. 

Our definition of obligation games opens the way for the definition of a probabilis- 
tic /i-calculus. Studies of such yu-calculus could complement studies of p-automata. In 
particular, issues such as nominals and counting connectives are usually studied in the 
context of logic. 

Finally, many questions regarding the theory of p-automata remain open. For in- 
stance, understanding the different transition modes of such automata (i.e., alternation 
vs. nondeterminism vs. determinism) and conversions between the different modes. 
A related question is that of feasibility of algorithmic questions such as emptiness of 
p-automata, which generalizes the satisfiability problem of pCTL. 
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